While consumers may access media items, such as movies and television shows, by receiving over the air signals by subscribing to a cable or satellite television provider, increasingly consumers are accessing more content over Internet-based media distribution systems. Some Internet-based media systems allow users to stream content over the Internet to a variety of client devices. For example, a streaming media system may provide content to users via a personal computer, a set-top box, or a personal mobile device, such as a smart phone or tablet computer. Streaming media systems enable users to access media content in a stream, such that the users may begin consuming (e.g., watching and/or listening to) content before the entirety of the content is delivered to the user's client device. Such a system allows users to access content while avoiding a potentially lengthy download process.
Such a system also allows for experimentation in the presentation of user interface elements to current and potential users. The scalability of such systems can provide large amounts of data to be collected in order to try to increase the accuracy of any prediction models to be derived from such experimentation. However, the volume of data available and needed to make accurate predictions can present a substantial computational load that is difficult or impossible to satisfy within the amount of time available for acting on such predictions.
Accordingly, the process assessing such experimental deployments in network-based distribution systems is not satisfactory in all respects.